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This paper discusses the formulation and execution of a laboratory test of the electrical 
interfaces between multiple atmospheric scientific instruments and the spacecraft bus that 
carries them. The testing, performed in 2002, used engineering models of the instruments 
and the Aura spacecraft bus electronics. Aura is one of NASA’s Earth Observatory System 
missions. The test was designed to evaluate the complex interfaces in the command and data 
handling subsystems prior to integration of the complete flight instruments on the 
spacecraft. A problem discovered during the flight integration phase of the observatory can 
cause significant cost and schedule impacts. The tests successfully revealed problems and 
led to their resolution before the full-up integration phase, saving significant cost and 
schedule. This approach could be beneficial for future environmental satellite programs 
involving the integration of multiple, complex scientific instruments onto a spacecraft bus. 


I. Introduction 

T he EOS Aura mission realized substantial cost and schedule benefits from early pre-integration testing of 
instrument engineering models with the spacecraft Software Development and Validation Facility (SDVF), a 
test bed consisting of engineering models of the spacecraft avionics. Earth Observatory System (EOS) Aura is the 
third in a series of low Earth-orbit environmental observatories developed for NASA’s EOS Program. Terra, which 
launched in December 1999, Aqua, which launched in May 2002, and Aura, which launched in July 2004, together 
provide an unprecedented global earth science dataset for the land (Terra), water (Aqua), and atmosphere (Aura). 
Aura and Aqua were both built on a common spacecraft bus that provides services including power, thermal control, 
attitude control, data collection and formatting, and communication. Each observatory carries a suite of complex 
instruments that monitor various facets of the Earth’s surface and atmosphere. Aura has four instruments to measure 
ozone and trace atmospheric gases: the Microwave Limb Sounder (MLS), the High Resolution Dynamics Limb 
Sounder (HIRDLS), the Tropospheric Emission Spectrometer (TES), and the Ozone Monitoring Instrument (OMI). 
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Each scientific instrument was designed, built, and tested by organizations other than the spacecraft contractor, and 
the spacecraft contractor was responsible for overall observatory integration. 

The complexity of merging multiple Command and Data Handling (C&DH) systems presents a major 
challenge for integrating the instruments onto the spacecraft. The spacecraft C&DH system provides command and 
telemetry distribution, science data handling, onboard data storage and timing services for the instruments. 
Instrument C&DH systems are independently developed and typically tailored to gather particular scientific data 
sets. Data handling interface issues may be inadvertently created through processor selection, independently 
developed software module dependencies and timing, or instrument sequence or scan-rate dependent timing. More 
subtle data handling issues may arise through general timing incompatibilities, packet sizing requirements, or 
interaction between multiple data streams. Further complications are also possible when trying to accommodate 
multiple instrument-unique database structures or schema. The overall design of the observatory involves extensive 
and often complex interfaces between the instruments and the spacecraft. 

For Aqua and Aura, the initial interface design and planning was done through a series of Interface Control 
Working Group meetings with agreements documented in formal Interface Control Documents (ICDs). Although 
very detailed in nature, capturing all the subtleties, complexities, and uncertainties in the ICDs is difficult. In the 
next phase of the Aqua interface development process, instrument interface testing was performed with the 
Spacecraft/Instrument Interface Simulator (SIIS). 

Limitations in ICD documentation and SIIS testing became apparent when C&DH issues and anomalies 
arose during Aqua observatory-level Integration and Test (I&T). During I&T of the Aqua Observatory, many 
incompatibilities were revealed in the details of the specific interfaces between the C&DH systems of the 
instruments and the spacecraft. There were problems in C&DH formatting, timing, signal levels and functional 
incompatibilities between the spacecraft Formatter/Multiplexer Unit/Solid State Recorder (FMU/SSR) and 
instrument 1553 and Transparent Asynchronous Transmitter-Receiver Interface (TAXI) data. These issues were not 
uncovered in the ICD development process (which does not address the design at a low enough level in the specifics 
of the data interface) nor were the problems revealed by the SIIS testing. The SIIS testing only addressed the 
simpler wiring and power interface levels and was not intended to have the signal-specific characteristics of the 
instruments required to validate the interfaces where the issues arose. Identifying and resolving these problems 
caused schedule delays during the Aqua Observatory I&T phase. Further, flexibility for addressing problems at this 
point was extremely limited because the instruments had already been delivered and installed, and fixes to the 
instruments themselves would have resulted in severe cost and schedule impacts. As a result, most fixes were 
implemented in the spacecraft software. 

Further complicating the Aqua observatory integration effort was the enormous task the spacecraft team 
undertook manually converting instrument databases, automated test procedures, and instrument telemetry displays 
to the EPOCH**-based spacecraft I&T software. A substantial learning curve by the instrument teams was also 
required to adapt to the new test software environment. 

Lessons learned from the Aqua experience coupled with the availability of both instrument and spacecraft 
engineering hardware provided the Aura program a unique opportunity to implement a series of pre-integration cost 
and schedule risk reduction activities. Each instrument team had an instrument Engineering Model (EM) or 
surrogate instrument simulator that could be made available. The Aqua experience demonstrated that the SIIS 
testing was insufficient for verifying instrument/spaceeraft interfaces prior to actual integration, thus for Aura, the 
team decided to forgo SIIS testing and conduct higher fidelity interface testing using the spacecraft SDVF facility 
and instrument engineering hardware. 


II. Test Concept 

A multi-faceted approach was adopted for spacecraft/instrument pre-integration testing, not only to pre- 
integrate instrument and spacecraft EM hardware, but to also involve and train instrument personnel in preparation 
for the upcoming observatory-level I&T. Several activities between the various instrument and spacecraft teams 
needed to be coordinated in order to support SDVF testing of the instrument engineering hardware. Instrument 
teams took responsibility for the development of instrument databases, automated test procedures, and telemetry 
displays ultimately used in observatory I&T. A Command and Telemetry Database Working Group was established 
for modifying the existing Aqua database schema to accommodate all required features of the Aura instrument 
databases. Training was provided to the instrument teams on the use of spacecraft EPOCH I&T software, 


** EPOCH is the name of an Integral System’s Inc. satellite ground system software product. 
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generation of EPOCH Satellite Test and Operations Language (STOL) procedures, and generation of telemetry 
displays. EPOCH workstations were provided to each instrument team for on-site training, procedure and display 
development, and initial validation of converted instrument databases, procedures, and telemetry displays. Each of 
these activities was essential and supported the goal of pre-integrating and testing the instrument and spacecraft 
engineering hardware in an environment that closely resembled the planned observatory I&T configuration. An 
overview of these activities and required interactions of the contributing organizations is shown in Figure 1 . 
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Uncovering interface issues during observatory-level integration can introduce significant delays. These 
delays can be very costly when a large I&T team is in place. The EOS Aura team developed this approach to testing 
interfaces as a way to reduce risk and save cost and schedule by identifying and correcting interface 
incompatibilities prior to full-scale observatory I&T. A test plan was developed to exercise key elements of critical 
interfaces, demonstrate flight software operation and 1553 bus loading, evaluate compatibility between the 
spacecraft Electrical Ground Support Equipment (EGSE) and Instrument Ground Support Equipment (IGSE), and 
exercise the command and telemetry data bases prior to electrical integration of instruments on the observatory. 


III. Test Planning 

Planning for the test required extensive coordination and communication amongst the various teams involved in 
the activities shown in Figure 1. Before detailed planning could begin, the scope of the test needed to be defined 
and goals established. Also, logistical issues needed to be identified and addressed. 

A. Test Scope 

The scope of the test was deliberately limited to six weeks to provide a deterministic and focused period for the 
team as well as contain costs. The six-week period included one week for each of the instrument teams to conduct 
standalone testing, one week for full up testing with all four instrument EMs operating, and one week for any 
additional tests identified in the previous weeks. 
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B. Test Goals 

Test goals focused on exercising the basic C&DH communication paths required for instrument integration. The 
first goal was to generate telemetry to verify several functions were working properly (e.g., electrical interfaces 
between the SDVF and instrument EMs, telemetry timing, data multiplexing in the telemetry stream, and de- 
commutation on the instrument telemetry display). The second goal was to send all command types possible. The 
third goal was to perform memory loads to the instruments; commands the spacecraft C&DH system stores and 
executes in a particular sequence. The fourth goal was to test the unique interface between the spacecraft Inertial 
Reference Unit (IRU) and the TES instrument. The fifth goal was to checkout the IGSE and EGSE command and 
data paths, which were different from what were used in the instrument I&T environments. The EGSE also stored 
and played back data in a file format similar to that used for nominal spacecraft operations. Spacecraft playback 
data would be sent to the instrument teams’ facilities to check their ability to receive and process it. 

C. Paper Audit of Interfaces 

A paper audit was conducted to check two types of interfaces that could not be verified in the SDVF; the passive 
analog interfaces for telemetry such as temperature and the power interfaces. The goal of the paper audit was to 
ensure that the expected electrical interface configuration matched the as-built configuration. The expected 
configuration was documented in each instrument’s ICD and the as-built configuration was documented in the latest 
version of the schematics. 

D. Logistical Considerations 

Several logistical issues were identified and addressed in preparation for testing. The SDVF was primarily used 
to checkout spacecraft software for both Aqua and Aura. Its use for several weeks would possibly impact ongoing 
software development. Program Management allocated six weeks where the SDVF would be dedicated to this 
testing while other, lower-fidelity simulators, were to be used for nominal Aqua and Aura software activities. 
Likewise, the instrument EMs and IGSE were being used by the instrument teams at their facilities for ongoing 
instrument development activities. The testing needed to be coordinated in such a way to avoid impacting flight 
instrument deliveries due to temporary unavailability of this equipment. 

Next, the SDVF needed to be reconfigured to support testing. To create the highest-fidelity test environment 
possible, EMs of the spacecraft FMU/SSR and IRU were added. Harnesses and cables to interface the instrument 
EMs to the SDVF were built to the same electrical requirements as those used for flight to match the electrical 
configuration on the observatory as closely as possible. 

Further, instrument databases for commands and telemetry were modified on an accelerated schedule to support 
testing. Accelerated development included development of procedures for testing as well as telemetry pages. 


IV. Test Set Up 

Figure 2 is a schematic of the test set up, showing the SDVF, the instrument EMs, the IGSE, and the spacecraft 
EGSE. 

A, Hardware 

The SDVF was modified to include the EMs of the spacecraft FMU/SSR and IRU. A network monitor (1553 
test set) was added to provide a method of displaying instrument housekeeping data, allowing for continuous data 
storage. A Science Data Distribution Unit (SDDU) was connected to the FMU/SSR to receive and distribute the 
real-time science data and provide a limited amount of data storage. 

The IGSE sets used for this test were the same that would be used for observatory I&T, though some were 
modified to account for differences between the flight instruments and their EMs. Each set of IGSE supplied power 
to the instrument EMs and received the science data streams. 

The MLS EM was not a full fit, form and function replica of the flight instrument, but a surrogate that provided 
simulated data streams where flight hardware was not replicated. For the OMI, a full fit, form and function EM of 
the Interface Adapter Module (IAM) was used. The IAM provided all the C&DH interfaces between the OMI and 
the spacecraft. Since no EMs of the science data gathering portions of the OMI were available, simulated OMI 
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Figure 2 SDVF Instrument Interface Test Configuration 


science and housekeeping data were used as input to the IAM EM. For the TES instrument, a full fit, form and 
function EM of its Instrument Electronics Module (IEM) was used with simulated science data as input. The 

HIRDLS EM was a full fit, form and function replica of the flight instrument. 

One challenge was accommodating several methods of providing power to the EMs while ensuring grounding 
remained isolated from the facility ground except at a single point. This single point ground was important in order 
to avoid ground loops that could perturb test results. Several modifications to the test setup were required to ensure 
proper grounding. 

B. Fidelity 

Since the harnesses were designed and built to provide the same point-to-point interfaces as the flight 
configuration, they provided a high-fidelity simulation of the spacecraft environment. The network interfaces for 
the test were not configured exactly the same as on the spacecraft, but a review of the network configuration and the 
electrical properties was performed to ensure there were no significant differences. 

The SDVF configuration with the added FMU/SSR and IRU EMs, and a new harness provided a high-fidelity 
simulation of the network or bus traffic that would occur on the observatory. This provided high confidence that 
problems found during SDVF testing would be representative of what would be expected to occur once the 
instruments were integrated onto the observatory. 

Use of the SDDU and playback of stored data also provided high fidelity simulation of the data that would be 
generated during observatory I&T. 


V. Test Program 

A test readiness review (TRR in Figure 1) preceded the start of the test program where, to the extent possible, the 
elements required to support testing were reviewed and ensured to be ready. Many problems surfaced during the 
test program, and most were resolved prior to the completion of the six-week test period. All problems were 
documented with the same tracking system in place for observatory I&T. Some of the problems were transitioned to 
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the Discrepancy Report (DR) system used by the spacecraft developer and the problem-tracking systems used by the 
instrument teams with the objective of resolving them prior to flight instrument integration. 

A. MLS Stand-Alone Testing 

The first instrument EM to be integrated in the SDVF and tested was the MLS surrogate. The MLS surrogate 
had several unique characteristics relative to the MLS flight unit and other instruments. The surrogate had a 
simulated serial interface to compensate for several instrument components. The surrogate also had a memory stick, 
which could be easily removed and used to modify the boot code. The same modifications on the flight unit 
required memory loads, which took several days to accomplish. Some solutions to problems uncovered during 
testing were implemented during the last week of testing. 

The MLS surrogate uncovered several problems with the SDVF that needed to be resolved before MLS testing 
could get underway. A total of 22 problems were found, the most of any instrument EM testing. Eight of the 
problems would have likely been resolved prior to flight instrument integration; for example, a patch to the ground 
system software was necessary to generate correct checksums on the commands sent to the instruments. Also, a 
patch was required to the spacecraft software to correct the spacecraft identifiers. However, there were 14 problems 
that would not have been detected until flight instrument integration. Major problems included a wrong 1553- 
connector type, incompatible telemetry, instrument resets caused by ancillary data packets and stale telemetry due to 
a unique telemetry format. 

The SDVF environment allowed for several workarounds without risking damage to the flight instrument. 
Workarounds included changes to the database, 1553 adapter cables and telemetry gathering to address telemetry 
generation problems. Most problems were successfully resolved prior to instrument integration and included 
modifications to instrument software, the database, the ground system software and the 1553 connector. 

B. HIRDLS Stand-Alone Testing 

The HIRDLS testing went very well. A few commands had the wrong Application Process Identifiers (APIDs) 
but were easily modified. A more significant problem that surfaced was incorrect generation of function codes. 
This required modifications to the instrument database before memory loads could be tested. The database was 
updated the next week to allow for testing during the fiill-up test. A spacecraft software patch was also implemented 
to correct an ancillary packet header problem. 

Testing revealed that memory loads were too large for single on-orbit contact, which required modifications so 
they could be divided. SDVF testing also provided experience in generating memory loads and led to development 
of a more efficient process. 

The Stored Command Sequence (SCS) testing uncovered an improper command structure, but the SCSs were 
easily modified prior to flight instrument integration. 

C. OMI Stand-Alone Testing 

The OMI IAM EM testing was the first to exercise the point-to-point TAXI science data interface and revealed a 
connection to the wrong port. This was a minor problem, but it required reconfiguration of the FMU/SSR and 
modifications to the procedures and SCSs. These changes were completed long before flight instrument delivery. 
Had this problem been revealed during flight instrument I&T, it would have caused significant cost and schedule 
impacts. 

Also, the instrument memory load test failed. The spacecraft software rejected commands due to sequence bits 
in the packet header. A patch to correct this was installed the next week and successfully resolved the problem. 

A couple of modifications to the instrument software were implemented as a result of the testing. First, post- 
processing of the data collected during the test uncovered a problem with the telemetry and led to modification of 
the IAM software. Next, the time tag in the packet header had an error, which was subsequently corrected. 

D. TES Stand-Alone Testing 

Testing with the TES IEM revealed the same telemetry problem found during the MLS testing. Taking 
advantage of the MLS experience, the software was updated the same day using a remote terminal. 

The TES instrument has a unique interface with the spacecraft IRU. This was tested using the IRU EM and 
revealed characteristics that would require retesting at the spacecraft level in order to verify proper waveforms. The 
data was received without errors, but was much noisier than expected. This test allowed an early understanding of 
the characteristics of this type of IRU data and led to close consideration of how the instrument software would 
handle it. This resulted in changing the TES flight software to provide filtering that would otherwise have been 
provided by the IRU. 
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Two other software modifications were made. First, the value of the engineering packet header length was 
corrected. Next, the software was modified to process the ancillary data packet properly. 

A couple of IGSE problems also surfaced. First, the Ethernet interface occasionally dropped packets. This was 
tracked to an incorrect initialization value, which was subsequently corrected. Second, the IGSE rejected all the 
ancillary packets. This problem was due to packet header information and was corrected with a spacecraft software 
patch. 

Several database problems were also uncovered. Some commands had the wrong APIDs, but were quickly 
corrected. Also, telemetry conversions were missing during the SDVF testing, but corrected with an updated 
database. 

E. Full-Up Testing 

The full-up testing, with all four instrument EMs operating simultaneously, went very well. In addition, a few 
open test goals from the stand-alone tests were completed during this test period. 

A two-hour tape of the real-time data was made for processing at the EOS Data Operations Service (EDOS). 
This allowed for early verification of the processing software. This tape also contained two hours of additional test 
data to evaluate EDOS processing of alternate virtual channel identifiers in the downlink. The data processing 
capability was verified, though some problems were found with data generation. 

The most significant problem found during full-up testing was that while the MLS, TES and OMI EMs were 
generating science data simultaneously with instrument housekeeping data, an overflow occurred in the FMU/SSR, 
causing loss of TES science data. The I&T configuration was designed to include, along with science data, 
housekeeping telemetry in the data stream to the IGSE sets. This provided a backup means for obtaining 

housekeeping data, which is included in the nominal EPOCH-system command and telemetry stream. Normally, a 
flight instrument would be powered off if command and telemetry data were interrupted since the health and safety 
of the instrument would be uncertain. The goal was to have this alternate housekeeping telemetry path available to 
avoid recycling instrument power, which could take up to five hours, should the command and telemetry data be 
interrupted. To complete SDVF testing, the housekeeping data was removed from the science data stream to the 
IGSE, since the need for a redundant path was not critical with the EMs. 

The full-up testing highlighted the FMU/SSR limitation and demonstrated that with only science data in the 
downlink to the IGSE sets, there was sufficient margin to avoid FMU/SSR overflows. Following the SDVF full up 
testing, analysis and a re-creation of the problem on the SDVF using simulated instrument inputs confirmed the 
FMU/SSR throughput problem. For observatory I&T, the instrument housekeeping data was removed from the 
IGSE data streams except when collection of science data was not critical (e.g., during communication link testing). 
For observatory thermal-vacuum testing, a 1553 monitor was added to provide a redundant means of obtaining 
housekeeping data. Luckily, the EPOCH system command and telemetry data was never interrupted during 
observatory I&T and the need to recycle power to the instruments was avoided. However, the HIRDLS and MLS 
instruments required that housekeeping data be provided to their IGSE sets for the purpose of processing their 
science data. This was accomplished via an Ethernet connection from the EPOCH system. Though this did not 
provide redundancy for determining instrument health and safety, it did provide the data necessary for processing 
their science data properly. Fortunately, this problem was identified early enough that the modifications required to 
address it did not impact observatory-level I&T. Time spent in the SDVF saved several weeks of troubleshooting at 
the observatory level. 


VI. Results 

Throughout the SDVF testing, the advantage of being in an EM environment was demonstrated. For example, 
there were several occasions where instrument software modifications were implemented without risk of damage to 
the EM or spacecraft components. The accessibility of the instrument EMs and SDVF components allowed for 
testing which would have been very time consuming and much more risky at the observatory level with flight 
hardware. 

Having dear and concise test goals helped to maintain focus and keep ihe test program progressing according to 
schedule. An incredible and cohesive team effort ensured that these goals were met. The contributions of spacecraft 
software, database, and each of the instrument teams were especially significant. 

The majority of problems found during this SDVF testing were resolved prior to spacecraft integration. This 
significantly contributed to the ability to maintain the flight instrument integration schedules once the instruments 
were delivered to the observatory I&T facility. The advantage of the having been through the SDVF testing was 
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evident in the maturity of the telemetry pages, command and telemetry databases and procedures at the time of each 
instrument’s integration onto the observatory. In addition, good communication paths were well established 
amongst the various organizations that needed to work in a coordinated fashion during observatory I&T . 

Integration at the observatory level still uncovered several problems that were not evident during SDVF testing. 
A timing problem that was only revealed after many hours of operation with the OMI IAM was found. Discovering 
this timing problem during SDVF testing would have been very unlikely due to the length of time required to cause 
it to occur. Another problem with the TAXI interface was not found until integration of the OMI IAM onto the 
spacecraft. This problem was traced to incorrect resistor values internal to the IAM. The TES IRU interface also 
had a timing problem that was evident in the pointing telemetry from the instrument’s mechanisms. This would not 
have been revealed during SDVF testing because there were no mechanisms in the TES IEM. 

Even with the few problems that did surface during observatory I&T, it is clear that significant delays were 
avoided due to the problems that were uncovered during SDVF testing and subsequently corrected. A summary of 
the planned versus actual I&T durations for the Aura instruments is shown in Table 1. 


Table 1. EOS Aura Planned vs. Actual Electrical Instrument Integration 



HIRDLS 

MLS 

OMIS 

TES 

Total 

*PIanned 

Integration 

(Shifts) 

15 

15 

15 

15 

60 

*ActuaI 

Integration 

(Shifts) 

15 

20 

8 

18 

61 

Delta 

(Shifts) 

0 

-5 

+7 

-3 

-1 


*Two shifts per day. 100 head average. 


VII. Conclusion 

Testing complex instrument-bus interfaces with engineering models of both the spacecraft bus C&DH system 
and the instrument units proved effective in identifying and correcting problems in advance of actual integration of 
the instruments on the spacecraft. Data transfer problems associated with timing, data bus resource management, 
packet characteristics, and multiple instrument interference proved difficult to control by conventional analytical 
techniques and documentation. The engineering models proved effective in finding and correcting the interface 
problems at a fraction of the cost that would have been borne had the observatory team waited until delivery and 
installation of the complete instruments. 
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